Real-time video processing technology can be used to provide high quality video conferencing and other interactive environments. For example, video conferencing systems can be used to enable interactions between two or more participants at remote locations. Signal processing techniques can be used to enhance the user experience while participating in a video conference or in other real-time video applications. Bandwidth constraints can limit the amount of data that can be used when distributing a given bandwidth budget to multiple conferencing users. As an example, some techniques sacrifice quality to compensate for a system load when multiple users share a common real-time communication channel.